Skip to content

Optimize file I/O by caching pyproject data during first pass#81

Merged
pontemonti merged 2 commits intousers/johanb/Dependenciesfrom
copilot/sub-pr-80
Dec 3, 2025
Merged

Optimize file I/O by caching pyproject data during first pass#81
pontemonti merged 2 commits intousers/johanb/Dependenciesfrom
copilot/sub-pr-80

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Dec 3, 2025 •
edited
Loading

Addresses review feedback about inefficient file I/O in generate_dependency_diagram.py where each pyproject.toml was read twice—once for package names, once for dependencies.

Changes

  • Added pyproject_data_cache dictionary to store parsed pyproject data during the first pass
  • Second pass now uses cached data instead of re-reading files
# Before: reading file twice
for path_str, pkg_type in package_configs:
    pyproject_data = read_pyproject_toml(pyproject_path)  # first read
    pkg_name = pyproject_data['project']['name']
    ...

for pkg in packages:
    pyproject_data = read_pyproject_toml(pkg.path)  # second read
    pkg.dependencies = extract_dependencies(pyproject_data, all_package_names)

# After: cache and reuse
pyproject_data_cache: Dict[str, Dict] = {}

for path_str, pkg_type in package_configs:
    pyproject_data = read_pyproject_toml(pyproject_path)
    pyproject_data_cache[pkg_name] = pyproject_data  # cache it
    ...

for pkg in packages:
    pyproject_data = pyproject_data_cache[pkg.name]  # use cache
    pkg.dependencies = extract_dependencies(pyproject_data, all_package_names)

Two-pass approach retained since all package names must be known before filtering internal dependencies.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI mentioned this pull request Dec 3, 2025
Merged
@pontemonti
Optimize file I/O by caching pyproject data during first pass
45cbfaf 
Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>
Copilot AI changed the title [WIP] Update dependency diagram and generation script based on feedback Optimize file I/O by caching pyproject data during first pass Dec 3, 2025
Copilot AI requested a review from pontemonti December 3, 2025 17:18
@pontemonti pontemonti marked this pull request as ready for review December 3, 2025 17:19
@pontemonti pontemonti requested a review from a team as a code owner December 3, 2025 17:19
@pontemonti pontemonti merged commit c8601ae into users/johanb/Dependencies Dec 3, 2025
1 check passed
@pontemonti pontemonti deleted the copilot/sub-pr-80 branch December 3, 2025 17:19
pontemonti added a commit that referenced this pull request Dec 4, 2025
@pontemonti
Add dependency diagram and script to generate dependency diagram. (#80)
a33e81a 
* Add dependency diagram and script to generate dependency diagram.

* Change layout of diagram to make it easier to read

* Update generate_dependency_diagram.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Optimize file I/O by caching pyproject data during first pass (#81)

* Initial plan

* Optimize file I/O by caching pyproject data during first pass

Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>

* Use regex for dependency version specifier parsing (#82)

* Initial plan

* Use regex for parsing dependency version specifiers

Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>

* Formatting fixes

---------

Co-authored-by: Johan Broberg <johanb@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@pontemonti pontemonti pontemonti approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@pontemonti pontemonti

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pontemonti